Skip to content

Conversation

@nik9000
Copy link
Member

@nik9000 nik9000 commented Mar 14, 2025

Adds the original_types to the description of ESQL's unsupported fields. This looks like:

    {
      "name" : "a",
      "type" : "unsupported",
      "original_types" : [
        "long",
        "text"
      ]
    }

for union types. And like:

    {
      "name" : "a",
      "type" : "unsupported",
      "original_types" : [
        "date_range"
      ]
    }

for truly unsupported types.

This information is useful for the UI. For union types it can suggest that users append a cast.

Adds the `original_types` to the description of ESQL's `unsupported`
fields. This looks like:
```
    {
      "name" : "a",
      "type" : "unsupported",
      "original_types" : [
        "long",
        "text"
      ]
    }
```
for union types. And like:
```
    {
      "name" : "a",
      "type" : "unsupported",
      "original_types" : [
        "date_range"
      ]
    }
```
for truly unsupported types.

This information is useful for the UI. For union types it can suggest
that users append a cast.
@nik9000 nik9000 added auto-backport Automatically create backport pull requests when merged :Analytics/ES|QL AKA ESQL ES|QL-ui Impacts ES|QL UI v9.0.0 v9.1.0 labels Mar 14, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Mar 14, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/kibana-esql (ES|QL-ui)

@elasticsearchmachine
Copy link
Collaborator

Hi @nik9000, I've created a changelog YAML for you.

@costin
Copy link
Member

costin commented Mar 14, 2025

Good idea.
As there might be more type information than the original type (such as recommended casting, potentially source indices, etc..) how about adding a dedicated section such as type_info to allow extensability:

    {
      "name" : "a",
      "type" : "unsupported",
      "type_info" : {
           "original_types": [ "date_range"]
      }
    },
   // aspirational
    {
      "name" : "a",
      "type" : "union",  // <-- different from unsupported
      "type_info" : {
           "original_types": [ "text", "long"],
           "safe_cast": ["text"]
      }
    }

@costin costin removed the v9.0.0 label Mar 14, 2025
* be actually supported types.
*/
@Nullable
public List<String> originalTypes() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Attribute != Field
This information makes sense only in the case of unsupported fields (potentially union types in the future) and thus makes sense on FieldAttribute & co (such as UnsupportedAttribute) alone.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean.

@costin
Copy link
Member

costin commented Mar 14, 2025

P.S. Removed 9.0 since it has the same restrictions as 8.18 - did you mean 8.19?

@nik9000
Copy link
Member Author

nik9000 commented Mar 14, 2025

P.S. Removed 9.0 since it has the same restrictions as 8.18 - did you mean 8.19?

That makes sense. Let's leave it out.

@nik9000
Copy link
Member Author

nik9000 commented Mar 14, 2025

I don't know if we can change the type from unsupported to union. It'd be cool at add a recommended cast.

@nik9000
Copy link
Member Author

nik9000 commented Mar 19, 2025

@costin do you think I should change the format to be:

    {
      "name" : "a",
      "type" : "unsupported",
      "type_info" : {
           "original": [ "text", "long"]
      }
    }

In this PR?

Edit: I didn't make this change as part of this PR.

@nik9000 nik9000 removed the auto-backport Automatically create backport pull requests when merged label Mar 19, 2025
@nik9000
Copy link
Member Author

nik9000 commented Mar 21, 2025

@costin I've pushed a change that moves how we communicate the original types out the rest API. I didn't change the json layout yet. That's a little trickier - but still fine. Just a bit trickier.

@nik9000
Copy link
Member Author

nik9000 commented Mar 21, 2025

If you want me to change the layout I can, but unless you really want it I think I won't.

@nik9000 nik9000 requested a review from costin March 21, 2025 17:08
@nik9000
Copy link
Member Author

nik9000 commented Mar 25, 2025

I talked with @costin last night and he approved this if I went back to the first way I was flowing the unsupported types through the system. We don't really agree on the right way to do this, but that's how things are when you build stuff together. Compromise.

I've pushed that and will set this to auto-merge.

@nik9000 nik9000 added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Mar 25, 2025
@elasticsearchmachine elasticsearchmachine merged commit f097818 into elastic:main Mar 26, 2025
17 checks passed
@nik9000 nik9000 deleted the esql_report_original branch March 26, 2025 02:22
omricohenn pushed a commit to omricohenn/elasticsearch that referenced this pull request Mar 28, 2025
Adds the `original_types` to the description of ESQL's `unsupported`
fields. This looks like:

```
    {
      "name" : "a",
      "type" : "unsupported",
      "original_types" : [
        "long",
        "text"
      ]
    }
```

for union types. And like:

```
    {
      "name" : "a",
      "type" : "unsupported",
      "original_types" : [
        "date_range"
      ]
    }
```

for truly unsupported types.

This information is useful for the UI. For union types it can suggest
that users append a cast.
nik9000 added a commit to nik9000/elasticsearch that referenced this pull request May 1, 2025
Claims a transport version in main that we will use to backport elastic#124913
to 8.19.
nik9000 added a commit that referenced this pull request May 1, 2025
Claims a transport version in main that we will use to backport #124913
to 8.19.
nik9000 added a commit to nik9000/elasticsearch that referenced this pull request May 2, 2025
Adds the `original_types` to the description of ESQL's `unsupported`
fields. This looks like:

```
    {
      "name" : "a",
      "type" : "unsupported",
      "original_types" : [
        "long",
        "text"
      ]
    }
```

for union types. And like:

```
    {
      "name" : "a",
      "type" : "unsupported",
      "original_types" : [
        "date_range"
      ]
    }
```

for truly unsupported types.

This information is useful for the UI. For union types it can suggest
that users append a cast.
@nik9000 nik9000 added the v8.19.0 label May 2, 2025
@nik9000
Copy link
Member Author

nik9000 commented May 2, 2025

Backport to 8.19.0: #127641

nik9000 added a commit that referenced this pull request May 2, 2025
Adds the `original_types` to the description of ESQL's `unsupported`
fields. This looks like:

```
    {
      "name" : "a",
      "type" : "unsupported",
      "original_types" : [
        "long",
        "text"
      ]
    }
```

for union types. And like:

```
    {
      "name" : "a",
      "type" : "unsupported",
      "original_types" : [
        "date_range"
      ]
    }
```

for truly unsupported types.

This information is useful for the UI. For union types it can suggest
that users append a cast.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) >enhancement ES|QL-ui Impacts ES|QL UI Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.19.0 v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants